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Abstract 

This  paper  discusses  the  collaborative  research  methodologies  and  techniques  developed  by  the 
Naval  War  College’s  Decision  Strategies  Department  (DSD),  and  how  these  methods  and  techniques 
have  allowed  the  DSD  to  meet  the  diverse  research  requirements  of  senior  naval  and  government 
decision  makers.  The  Decision  Strategies  Department’s  “experts  at  talking  to  experts”  collaborative 
research  process  is  discussed,  as  are  some  of  the  techniques  the  department  employs  to  make  this 
research  approach  interesting  and  enjoyable  for  their  “decision  event”  participants,  as  well  as 
effective  at  generating  the  research  data  necessary  to  address  the  decision  maker  or  research  spon¬ 
sor’s  questions.  The  paper  describes  how  this  research  approach  evolved,  why  it  is  different  from 
traditional  research  or  war  gaming,  and  the  type  of  research  requirements  that  caused  this  approach 
to  be  developed.  The  paper  discusses  the  lessons  learned  from  using  this  research  process,  and  some 
of  the  ways  that  the  department  disseminates  the  results  of  its  work.  Also  discussed  are  two  DSD  re¬ 
search  projects  that  illustrate  different  approaches  to  this  type  of  collaborative  research,  and  that 
contain  examples  of  techniques  that  can  even  make  this  type  of  collaborative  research  fun! 

We  don’t  do  research  the  “old  fashioned  way” 

In  the  Decision  Strategies  Department  (DSD)  at  the  Naval  War  College  we  don’t  do  research  the 
old  fashioned  way.  From  our  point  of  view,  traditional  research  involves  various  combinations  of 
literature  reviews  (both  the  library  and  the  Internet  variety),  and  possibly  some  interviews,  or  a  sur¬ 
vey,  or  an  experiment,  or  a  conference,  the  results  of  which  are  analyzed  to  inform  the  research.  The 
normal  application  of  Group  Support  Systems  (GSS),  that  include  groupware  such  as  GroupSys- 
tems,2  is  to  help  an  existing  group  or  organization  reach  a  specific  decision,  or  work  through  a  series 
of  issues  usually  defined  by  the  group  itself.  In  the  Decision  Strategies  Department,  we  employ  some 
of  the  aspects  of  traditional  research,  and  we  use  GSS,  but  we  have  also  developed  what  we  feel  are 
unique  methods  of  conducting  collaborative  research.  In  short,  we  assemble  a  diverse  group  of  ex¬ 
perts  and  develop  a  “Decision  Event”  agenda  designed  to  elicit  from  these  experts  the  data  needed  to 
address  the  questions  and  issues  that  the  research  sponsor  has  requested  we  investigate.  Often  our 
approach  contains  some  aspects  of  war  gaming  in  that  we  employ  scenarios,  scene-setting  tech¬ 
niques,  and  role  playing.  Sometimes  our  role  playing  is  like  a  skit  where  participants  are  asked  to 
“respond  from  the  perspective  of  a  small  middle-eastern  country”  for  a  10  minute  data  collecting 
exercise,  and  then  to  respond  from  an  entirely  different  perspective  a  few  minutes  later.  In  other 
events  our  role  playing  is  more  “movie  like”  as  we  ask  participants  to  “assume  the  role  of  a  North 
Korean  diplomat”  for  a  whole  day  and  a  half  event.  DSD  uses  GroupSystems  groupware  as  our 


principal  data-gathering  tool,  but  we  use  a  variety  of  techniques  to  make  our  decision  events  interest¬ 
ing,  enjoyable,  and  effective.  This  paper  will  discuss  how  we  do  this. 

Imagine  the  following  situation.  You  have  been  invited  to  participate  in  a  daylong  conference  exam¬ 
ining  the  future  of  foreign  direct  investment  in  Asia  in  the  year  2010.  The  venue  sounds  interesting — a 
plush  conference  center  overlooking  the  Rocky  Mountains.  The  views  during  the  breaks  should  be  stu¬ 
pendous,  but  the  thought  of  spending  8  hours  discussing  foreign  direct  investment  with  a  bunch  of  Wall 
Street  financiers  has  you  wondering  if  the  view  will  be  enough  to  overcome  what  sounds  like  a  stupe¬ 
fying  day. 

Now  imagine  that  the  day  is  over  and  you  find  that  you  learned  something,  you  feel  your  contribu¬ 
tions  were  meaningful,  and  yes,  to  your  surprise,  you  enjoyed  yourself.  All  in  all  this  was  a  day  well 
spent. 

Researchers  who  rely  on  personal  interviews  or  direct  input  for  collecting  data  face  the  same  two 
problems  every  time  they' begin  a  new  project — how  to  get  participants  and  how  to  keep  them  inter¬ 
ested.  If  their  decision  matrix  requires  them  to  determine  more  than  “which  cola  do  you  like  better?” 
then  the  challenge  of  keeping  participants  energized  and  engaged  increases.  If  the  event  involves 
senior  subject  matter  experts,  then  convincing  them  to  spend  time  on  your  project  rather  than  invest¬ 
ing  it  in  a  myriad  of  competing  priorities  is  a  real  challenge.  If  you  do  everything  right,  the  payoff 
comes  at  the  end  of  the  day  when  participants  say  they  not  only  learned  something,  and  felt  they 
made  a  positive  contribution  to  your  research,  but  also  enjoyed  the  experience. 

We  learned  that  how  we  ask  questions  and  frame  issues  are  big  factors  in  whether  the  collabora¬ 
tive  research  experience  is  a  positive  one  for  participants,  and  we  believe  we  have  fostered  several 
unique  approaches  for  conducting  collaborative  research.  Those  approaches,  and  how  we  use  them  to 
prepare  participants  to  use  a  particular  GroupSystems  tool,  are  what  we  believe  makes  our  collabora¬ 
tive  research  unusual  and  fun. 

During  a  recent  lunchtime  event  for  a  group  of  40  admirals  and  senior  executive  service  govern¬ 
ment  civilians,  DSD  researchers  were  asked  to  conduct  a  short,  focused  brainstorming  session  on 
transforming  the  U.S.  Navy.  A  traditional  approach  might  have  involved  asking  the  admirals  to 
brainstorm  undirected,  and  then  pare  down  the  ideas  to  a  smaller  subset  through  discussion,  or  a  se¬ 
ries  of  votes  or  priority  rankings — a  time  consuming  and,  often,  disappointing  venture.  Realizing  that 
we  only  had  the  admirals  for  a  little  over  an  hour.  Dr.  Thomas  Barnett  and  the  DSD  team  led  the  par¬ 
ticipants  through  a  stimulating  discussion  using  a  very  different  process.  After  setting  the  stage,  the 
admirals  “played”  a  version  of  the  reality-based  television  show  Survivor.  The  “contestants”  were 
traditional  naval  warfare  areas  (e.g.,  long  range  strike,  littoral  warfare,  etc.),  and  they  were  “voted  off 
the  island”  using  a  series  of  “challenges”  (which  defined  the  decision  criteria).  Participants  (voters) 
decided  for  themselves  which  warfare  area  was  least  important  to  a  future  transformed  Navy.  Fol¬ 
lowing  each  vote.  Dr.  Barnett  led  a  verbal  discussion  that  encouraged  the  admirals  to  discuss  the  key 
attributes  of  each  warfare  area  and  why  they  thought  it  was  or  was  not  important  to  the  future  suc¬ 
cess  of  the  Navy.  At  the  end  of  the  hour,  the  admirals  had  a  new  appreciation  for  their  biases  and  the 
effect  they  could  have  on  a  transformed  U.S.  Navy.  They  also  had  an  idea  of  what  warfare  attributes 
(collectively  and  by  what  margin)  others  believed  were  essential  for  the  future  of  the  Navy.  Although 
a  subject  as  complex  as  Navy  transformation  requires  months  of  work,  this  technique  allowed  us  to 
quickly  broach  the  subject  and  get  input  from  committed  experts  while  the  participants  were  assem¬ 
bled  in  Newport  for  another  purpose.  Besides  fostering  an  interesting  discussion  that  quickly  cen¬ 
tered  on  difficult  trade-offs  that  must  be  made  when  an  organization  such  as  the  U.S.  Navy  trans¬ 
forms,  the  participants  enjoyed  themselves  and  did  not  regret  using  their  lunch  hour  to  tackle  a  diffi¬ 
cult  issue. 

How  this  research  approach  came  about 

The  Decision  Strategies  Department  (DSD)  is  a  relatively  new  department  at  the  Naval  War  Col¬ 
lege.  Formed  in  1995  to  operate  the  first  of  a  proposed  network  of  naval  decision  support  centers 


2 


(DSC)  for  senior  naval  leadership,  the  DSD  still  performs  that  function,  but  its  mission  has  evolved 
over  the  years.  The  department  is  small — consisting  of  a  chairman  and  deputy  chairman,  three  senior 
researchers,  a  technical  systems  manager,  and  a  secretary.  Occasionally  two  support  contractors 
augment  the  department,  one  in  the  research  area  and  the  other  in  the  technical  support  area.  Even 
though  the  staff  is  small,  it  consists  of  people  from  diverse  backgrounds  that  have  a  broad  base  of 
expertise. 

From  the  beginning,  the  staff  was  dedicated  to  directly  supporting  decision  makers  through  the  use 
of  decision  support  software  in  a  collaborative,  multi-media  GSS  environment  like  our  two-year-old 
facility  pictured  in  Figure  1. 

We  still  support  decision  makers  in  our  GSS  setting  as  they  grapple  with  specific,  well-defined  is¬ 
sues.  In  these  cases,  they  define  the  agenda  and  invite  the  participants,  and  we  facilitate  the  event 
using  verbal  and  computer  based  techniques.  Over  the  years,  however,  the  department  has  found  it¬ 
self  doing  more  and  more  broadly  based,  less  clearly  defined  research.  As  our  name  suggests,  our 
department  is  the  one  people  turn  to  when  the  issues  are  so  complex  the  requestors  are  not  even  sure 
what  questions  to  ask.  Our  approach  to  conducting  this  latter  type  of  research  has  evolved  into  vari¬ 
ous  methods  that  take  advantage  of  particular  staff  expertise,  as  well  as  our  GSS  skills,  to  conduct 
research  in  a  collaborative  venue.  We  like  to  think  of  ourselves  as  “experts  at  talking  to  experts.” 
DSD’s  research  methodology  stems  in  large  part  from  the  type  of  tasking  we  receive,  as  well  as  the 
goals  of  the  decision  maker  asking  for  the  research. 

Often  we  are  asked  by  senior  officials  to  answer  a  question  or  explore  an  issue  that: 

•  Relates  to  current  events  and  requires  a  rapid  response 

•  Requires  working  with  participants  from  a  diverse  array  of  disciplines,  areas  of  operational 
experience,  and  organizational  cultures 

•  Is  a  current  problem  that  needs  a  long-term  solution  or  strategic  plan  developed  quickly 


Figure  1:  Decision  Support  Center,  Naval  War  College 
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•  The  decision  maker  and  his/her  staff  are  too  busy  with  their  “in-box”  to  address 

•  The  leader  desires  a  different  or  fresh  point  of  view 

•  He/she  desires  to  develop  widespread  “buy-in”  for  the  ultimate  decision 

•  Provides  for  ongoing  updates  to  the  research  sponsor/decision  maker  as  information  becomes 
known,  as  well  as  the  opportunity  for  him/her  to  make  changes  in  the  questions  or  issues  be¬ 
ing  addressed  as  the  situation  changes 

•  Requires  broad,  forward-thinking  research  that  frames  future  security  issues  in  a  new  light 

The  Decision  Strategies  Department  has  conducted  this  type  of  research  for  a  number  of  senior 
military  and  government  officials  who  obviously  bring  their  own  perspective  to  the  research  and  de¬ 
cision  making  process.  We  believe  our  success  in  meeting  the  tasking  imposed  by  these  leaders  re¬ 
flects  positively  on  our  collaborative  research  approach,  and  the  process  techniques  we  have  devel¬ 
oped.  Recent  DSD  research  sponsors  include: 

•  The  Secretary  of  the  Navy 

•  The  Chief  of  Naval  Operations  (four  star  admiral) 

•  The  Commander  in  Chief  Pacific  (four  star  admiral) 

•  The  President  of  the  Naval  War  College  (three  star  admiral) 

•  The  admiral  in  charge  of  naval  aviation  construction  programs  (two  star  admiral) 

•  The  Commander  of  the  Navy  Warfare  Development  Command  (two  star  admiral) 

•  The  admiral  in  charge  of  all  naval  research  (two  star  admiral) 

•  The  United  Nations  Secretariat 

•  The  Office  of  the  Secretary  of  the  Navy 

Meet  face-to-face  to  understand  what  you  are  being  asked  to  research 

When  tasking  is  received  from  a  research  sponsor  we  have  learned  that  one  of  the  most  critical 
steps  is  to  have  the  DSD  project  director  meet  with  the  decision  maker/research  sponsor  personally. 
Depending  on  the  seniority  of  the  sponsor,  we  often  meet  several  times  with  a  member  of  the  spon¬ 
sor’s  staff  to  get  a  feel  for  what  the  issues/questions  are  that  we  will  be  researching.  Sometimes  we 
will  develop  a  “straw  man”  agenda  for  addressing  the  issues  and  answering  the  questions  the  sponsor 
has  raised.  The  DSD  project  director  will  then  discuss  this  notional  agenda  dining  the  meeting  with 
the  sponsor.  Without  a  face-to-face  meeting  with  the  person  who  is  asking  the  questions,  and  who 
will  be  receiving  the  output  of  your  research,  the  probability  is  high  that  he  or  she  will  be  less  than 
fully  satisfied  with  the  research  results.  Establishing  a  connection  with  the  decision  maker  early  on 
also  facilitates  future  modifications  in  the  direction  of  the  research,  as  well  as  ensuring  that  the  deci¬ 
sion  maker  gets  the  information  he  or  she  needs  when  they  need  it. 

Learn  background,  design  event,  sign-up  experts,  show  time 

Once  the  project  director  has  met  with  the  sponsor  and  is  confident  about  what  is  being  requested, 
DSD’s  research  approach  generally  begins  with  background  research  to  get  facilitators  up  to  speed  and 
to  illuminate  the  issue.  An  overall  study  design  is  established  (see  specific  examples  below).  The  par¬ 
ticipants  are  identified  and  invited  to  the  event(s).  This  information  is  then  used  as  the  backdrop  for  a 
collaborative  event  during  which  a  variety  of  group  facilitation  techniques  and  decision  support  group- 
ware  tools  are  used.  In  contrast  to  most  group  facilitation  and  GroupSystems  (groupware)  decision  sup¬ 
port  applications — where  the  group  composition,  and  the  meeting  agenda  are  largely  defined  by  the 
customer — for  our  more  broadly  defined  collaborative  research,  often  it  is  the  DSD  project  director 
who  defines  both  the  make-up  of  the  group  of  experts,  and  the  specific  agenda  the  group  will  follow 
during  a  particular  decision  event.  Finally,  data  collected  at  the  event(s)  are  analyzed  and  the  research 
sponsor/decision  maker  is  informed  of  the  results. 
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The  most  important  decisions  made  during  this  process  involve  the  development  of  relevant  ques¬ 
tions,  the  selection  of  appropriate  techniques  in  order  to  elicit  the  desired  data,  and  the  identification 
of  the  right  mix  of  participants.  The  success  of  our  collaborative  research  approach  hinges  not  only 
on  the  quality  of  the  participants,  and  the  level  of  their  expert  knowledge,  but  also  on  the  diversity  of 
the  background,  and  the  differing  perspectives,  of  the  participants.  Just  assembling  a  group  of  senior 
military  officers  to  address,  for  example,  alternative  Korean  futures,  is  unlikely  to  yield  many  new  or 
fresh  perspectives,  and  unlikely  to  educate  and  illuminate  the  participants  by  exposing  them  to  new 
or  opposing  points  of  view.  However,  if  the  decision  event  participants  also  include  a  representative 
of  an  international  religious  relief  organization  who  had  experience  living  and  working  in  North  Ko¬ 
rea,  and  a  social  scientist  or  two,  the  opportunities  for  real  inquiry  and  for  generating  new  ideas  are 
greatly  increased. 

As  mentioned  above,  the  process  is  completed  by  careful  analysis  of  data  gathered  during  the  deci¬ 
sion  event.  Some  projects  may  repeat  the  research-decision  event-analysis  cycle  several  times,  using 
subsequent  events  (refined  by  previous  events)  to  explore  different  aspects  of  the  research  agenda.  Ide¬ 
ally,  the  results  of  the  research  are  presented  in  a  culminating  decision  event  during  which  the  policy¬ 
maker,  and  his  or  her  staff,  are  exposed  to  the  same  issues,  in  the  same  context,  as  previous  event  par¬ 
ticipants.  This  helps  provide  the  decision  maker/research  sponsor  with  the  context  for  the  answers 
and/or  issues  being  presented.  When  possible,  key  participants  from  previous  events  are  present  to  pro¬ 
vide  continuity  as  well  as  answer  specific  questions  raised  by  the  research  sponsor.  Figure  2  illustrates 
some  of  the  steps  taken  when  designing  a  decision  event. 


Decision  Event  Design 


Basic  Research 


Research  Event 


Identify  issues 
Structure  questions 
Identify  Event 
participants 


Expand  on  Basic  Research 
using  expert  input 
Identify  strategies  or 
scenarios 


Follow-on  Event 


Decision  Event 


Given  previous  steps 
Hold  Event  for  Decision 
Makers 

Develop  Action  List,  Strategy 
or  Policv 


Given  Research  Event  or 
Scenario  Developed 
Validate  and  Refine  Research 
or  Scenario  with  new  Group 
Identify  Kev  Decision  Issues 


Figure  2:  DSD  Decision  Event  Design 
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Get  the  “word”  out 


The  Decision  Strategies  Department  places  a  high  priority  on  getting  the  results  of  our  research 
out,  not  just  to  the  decision  maker  who  requested  the  research,  but  also  to  anyone  who  can  benefit 
from  it.  When  possible,  this  is  initially  accomplished  by  the  decision  maker  event/briefing  discussed 
above.  We  also  publish  our  work  on  our  website  and  conduct  numerous  PowerPoint  and  “interac¬ 
tive”  briefings.  During  the  interactive  briefings,  we  replicate  key  aspects  of  the  original  decision 
event  (including  replicating  the  GroupSystems  votes,  list  building  exercises,  etc.)  so  that  the  people 
attending  the  briefing  get  a  feel,  not  only  for  the  results  of  our  research,  but  also  for  how  it  was  con¬ 
ducted.  These  interactive  briefings  afford  the  project  director  the  opportunity  to  gather  further  data, 
as  well  as  to  validate  some  of  the  findings  from  the  principal  research  event  held  with  the  subject 
matter  experts. 

The  results  of  our  Y2K  International  Security  Dimension  Project  were  presented  dozens  of  times  to 
audiences  from  the  President’s  Y2K  czar  John  Koskinen,  to  Deputy  Secretary  of  Defense  John  Hamre, 
to  a  variety  of  industry  associations  (banking,  credit  cards,  electricity).  Insights  gained  from  our  South 
Asia  (Nuclear  Weapons)  Proliferation  Project  were  briefed  to  the  country  teams  at  the  American  Em¬ 
bassies  in  India  and  Pakistan  as  well  as  to  the  Commander-in-Chief  of  U.S.  Military  Forces  in  the  Pa¬ 
cific.  We  also  take  advantage  of  the  opportunity  to  brief  the  dozens  of  dignitaries  that  visit  the  Naval 
War  College  each  year  on  what  we  do  and  on  the  results  of  our  recent  projects.  We  provide  them  with 
hard  copy  reports  and  our  web-site  address  if  they  show  interest  in  learning  more  details  about  a  par¬ 
ticular  project.  Additionally,  we  brief  journalists  and  other  media  people,  as  well  as  write  for  the  print 
media  when  the  opportunity  for  publication  is  present. 

Researchers  do  not  like  to  have  their  work  sit  on  a  bookshelf,  and  we  do  our  best  to  avoid  that  by 
having  a  research  sponsor  (if  possible)  who  has  asked  that  a  particular  subject  be  studied.  This  situa¬ 
tion  is  ideal  as  it  provides  a  decision  maker  who  is  looking  forward  to  hearing  what  you  have  to  say. 
Even  in  those  few  cases  where  the  research  is  self-initiated,  we  make  sure  it  is  relevant  and  timely, 
and  then  we  disseminate  the  results  of  our  research  as  described  above. 

South  Asia  Proliferation  Project  (SAPP) 

The  first  example  of  a  Decision  Strategies  Department  collaborative  research  strategy  is  the  South 
Asia  Proliferation  Project  (SAPP).  The  following  excerpt  is  from  a  recent  article  in  the  Naval  War 
College  Review 3  by  the  director  of  the  SAPP,  Ambassador  Paul  D.  Taylor.  Ambassador  Taylor  em¬ 
ployed  several  techniques  in  conducting  his  research,  including  developing  a  scenario  (which  he 
elaborated  over  the  course  of  the  project)  to  place  his  expert  event  participants  in  a  specific  context 
before  asking  them  to  respond  to  his  tasking  via  GroupSystems  and  in  verbal  discussion.  During  the 
various  decision  events  that  comprised  this  project,  Ambassador  Taylor  often  reviewed  the  Group- 
Systems  data  as  it  was  being  generated  anonymously  by  the  participants  in  response  to  a  specific 
tasking,  and  then  picked  out  interesting,  contradictory,  provocative,  or  salient  comments  for  verbal 
discussion.  In  events  such  as  this,  it  is  usual  for  the  GroupSystems  work  to  consume  from  10  to  40 
percent  of  the  time,  with  the  majority  of  the  participant's  time  being  spent  in  verbal  discussion  of 
pertinent  issues  and  ideas. 

“Thinking  about  the  unthinkable” 

Shortly  after  the  nuclear  tests  in  India  and  Pakistan  in  May  1998,  the  Decision  Strategies  Depart¬ 
ment  undertook  a  series  of  simulations  and  “decision  events”  designed  to  examine  the  consequences 
of  these  developments.  The  project  started  from  the  premise  that  the  tests  had  increased  the  possibil¬ 
ity  of  weaponization,  deployment,  and  use  of  nuclear  weapons  in  South  Asia.  In  the  tradition  of 
games  that  the  Navy  has  conducted  at  the  Naval  War  College  in  Newport  for  more  than  a  hundred 
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years,  the  college  gathered  experts  from  the  U.S.  government,  academia,  foreign  governments,  busi¬ 
ness,  private  voluntary  and  nongovernmental  organizations,  and  military  commands  to  react  to  a  pre¬ 
pared  scenario  set  in  the  year  2003.  Asking  people  from  diverse  backgrounds  and  organizations  to 
interact  with  one  another  as  they  grappled  with  the  issues  posed  by  a  hypothetical  scenario  was  in¬ 
tended  to  produce  insights  that  might  have  eluded  an  individual  researcher  or  a  group  working  within 
a  single  discipline.  In  addition  to  playing  roles  in  simulations,  participants  were  invited  to  develop 
their  views  in  seminars  employing  a  combination  of  anonymous  commentary  [using  GroupSystems 
installed  on  a  LAN  of  laptop  computers]  and  conventional  discussions.  The  scenario  depicted  con¬ 
flict  in  South  Asia  escalating  from  civic  unrest  and  terrorism  to  an  exchange  of  tactical  nuclear 
weapons;  the  events  ranged  in  length  from  four  hours  to  six  working  days. 

Major  issues 

In  the  interest  of  learning  how  the  players  representing  the  U.S.  government,  other  governments,  and 
other  actors  would  respond  to  the  hypothetical  events,  researchers  acting  as  game  controllers  presented, 
in  various  simulation  settings,  successive  segments  of  the  event  scenarios  and  then  gave  players  free 
rein  to  react  as  they  believed  their  “character”  would  in  a  real  situation.  In  seminars,  the  questions 
posed  were  deliberately  broad  and  open-ended,  such  as,  “How  do  you  think  your  organization  or  other 
organizations  would  respond  to  the  events  you  have  just  heard  described?”  [Enter  your  responses  in  the 
GroupSystems  Categorizer]  In  this  manner,  the  organizers  tried  to  avoid  constraining  responses,  as  well 
as  to  encourage  maximum  interactions  among  participants. 

The  game  scenario 

In  the  tradition  of  military  gaming,  a  difficult  scenario — the  overall  “scripted”  background  and  the 
situational  framework  within  which  role  playing  and  discussion  of  important  issues  were  to  proceed 
during  the  successive  events — was  elaborated.  Developed  in  consultation  [during  a  scenario  building 
decision  event]  with  experts  on  South  Asia  from  the  U.S.  government  and  academia,  the  scenario 
was  intended  to  stimulate  planners  to  address  tough  challenges.  The  designers  stressed  to  participants 
in  all  the  events  that  the  scenario  was  to  be  considered  as  only  one  possible  future  and  that  it  was  not 
intended  in  any  sense  as  a  prediction  of  the  most  likely  evolution  of  events.  The  scenario  was  essen¬ 
tially  the  same  in  each  event,  but  it  became  more  elaborate  as  the  series  evolved — especially  after  an 
event  known  as  the  International  Game,  when  the  actions  of  foreign  players  moved  the  narrative  far¬ 
ther  than  had  been  envisioned. 

International  perspectives 

In  January  1999,  researchers  organized  a  two-day  international  simulation  to  gauge  reactions  to 
these  hypothetical  events  and  the  capacity  of  the  international  community  to  prevent,  manage,  and 
resolve  such  a  conflict.  The  simulation  brought  together  a  multinational  cross  section  of  diplomats, 
academics,  analysts,  and  military  personnel.  The  countries  represented  were  Australia,  Canada, 
China,  Finland,  France,  India,  Iran,  Japan,  Pakistan,  Peru,  the  Philippines,  Russia,  Singapore,  the 
United  Kingdom,  and  the  United  States. 

Many  of  the  diplomats  present  had  reached  the  rank  of  ambassador  in  their  countries’  diplomatic 
services;  one  had  served  as  foreign  minister.  In  the  game,  they  were  given  roles  as  their  countries’ 
principal  representatives  in  the  UN  Security  Council;  in  fact,  they  had  also  to  replicate  their  govern¬ 
ments’  entire  decision-making  authority.  The  diplomats  worked  without  the  benefit  of  instructions 
from  home,  and — with  the  exception  of  India  and  Pakistan — their  actions  in  the  simulation  were 
“free  play,”  based  on  their  individual  best  judgment  of  what  their  respective  governments  would  do 
in  response  to  the  hypothetical  situation  as  it  unfolded.  The  scenario  prescribed  most  of  the  military 
actions  of  India  and  Pakistan;  however,  even  those  nations’  representatives,  who  were  experienced 
diplomats  and  scholars,  devised  their  own  diplomatic  activities. 
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Crisis  Phase  I 
7-19  July,  2003 


c-ec-40*  c- 


♦  Pakistan  frustrated  by  lack  of 
progress  since  1999. 

♦  Staged  inflammatory  response 
to  failure  of  new  UNSC 
Kashmir  resolution. 

♦  Upturn  in  anti- Indian. 
anti-Hindu  rhetoric. 

♦  Demonstrations  in  Islamabad, 
and  other  major  cities. 

♦  Sharp  upturn  in  violence 

in  Indian-controlled  Kashmir. 


Figure  3:  Sample  Scenario  Slide 


The  International  Game  was  unusual  in  having  countries  represented  by  their  own  nationals.  This 
arrangement  brought  a  greater  degree  of  reality  to  the  responses  of  foreign  countries  than  is  usual  in 
U.S.  government-sponsored  simulations,  which  are  generally  played  exclusively  by  American  ex¬ 
perts. 

It  should  be  noted  that  the  international  participants  readily  adapted  to  the  use  of  the  groupware, 
indicating  that  language  and  cultural  barriers  were  not  a  factor,  at  least  for  this  group.  A  real  world 
by-product  of  these  decision  events  is  that  often  a  number  of  participants  from  different  organiza¬ 
tions,  who  had  not  previously  known  each  other,  establish  contact  and  began  a  dialog  that  they  vow 
to  maintain  after  completion  of  the  event. 

DSD  researchers  prepared  written  reports  on  various  events  in  the  SAPP  series  and  personally 
briefed  key  decision  makers.  Just  prior  to  President  Clinton’s  trip  to  India  and  Pakistan  in  March 
2000,  The  Washington  Post  published  an  op-ed  piece  by  Ambassador  Taylor  on  the  principal  insights 
the  project  evoked. 

Newrulesets.project 

In  this  second  example  of  a  Decision  Strategies  Department  collaborative  research  strategy, 
NewRuleSets.Project  director  Dr.  Thomas  Barnett  employs  quite  a  different  set  of  techniques  to  fa¬ 
cilitate  his  series  of  economic  security  decision  events.  After  assembling  the  right  people  for  the  par¬ 
ticular  research  questions  being  explored.  Dr.  Barnett  employs  innovative  ways  to  “ask”  the  partici¬ 
pants  for  their  input.  The  following  is  an  excerpt  from  Dr.  Barnett’s  revised  Asian  Energy  Futures 
Decision  Event  Report :4 
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We  designed  the  Asian  Energy  Futures  decision  event  with  these  major  goals  in  mind: 

•  Generate  “new  maps”  of  global  energy  market  relationships  based  on  a  clearer  understanding 
of  the  developmental  challenges  faced  by  major  Asian  economies  over  the  coming  decade 

•  Delineate  the  key  scenario  variables  and  dynamics  likely  to  emerge  as  Asia’s  energy  needs 
balloon  in  the  coming  years,  focusing  on  possible  regional  flashpoints 

•  Construct  comprehensive  downstream  scenarios  capturing  both  the  regional  and  global  ad¬ 
justments  to  Asia’s  energy  expansion. 

The  event  involved  two  dozen  participants  drawn  equally  from  the  financial  community,  the  po¬ 
litical-military  community,  and  the  regional  expert  community. 

The  point  of  the  effort  was  not  to  amass  the  most  impressive  collection  of  energy  and  Asian  ex¬ 
perts,  but  to  bring  together  a  diverse  array  of  experts,  decision  makers,  and  opinion  leaders  from  both 
the  public  and  private  sectors,  and  let  the  synergy  of  their  intellectual  interactions  serve  as  the  fun¬ 
damental  analytic  output.  In  short,  this  event’s  calling  card  was  a  “clash  of  paradigms,”  and  not  a 
rigorous  forecasting  effort.  The  event  unfolded  over  four  major  sessions.  Each  session  involved  both 
facilitated  discussion  by  the  group  as  a  whole  and  individual  participation  in  collective  brainstorming 
tasks,  in  which  we  employed  a  decision  software  system  known  as  GroupSystems.  Using  GroupSys- 
tems,  each  participant  entered  ideas  anonymously  via  a  dedicated  laptop,  while  simultaneously  com¬ 
menting  on  each  other’s  inputted  ideas  asynchronously  via  a  portable  Local  Area  Network,  or  LAN.  In 
effect,  then,  we  interspersed  facilitated  discussion  with  a  LAN  equivalent  of  a  “chat  room”  where  we 
explored  numerous  specific  ideas  in  greater  detail. 

The  Asian  Energy  Futures  event  basically  explored,  over  four  substantive  sessions,  a  rough  “influ¬ 
ence  net”  model  that  we’ve  constructed  regarding  the  key  dynamics  of  Asia’s  energy  future  and  its 
impact  on  the  global  economy  and  security  environment: 
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•  Concerning  “The  Choice,”  [from  the  above  slide]  we  conducted  one  session  called  “You 
Make  the  Call!”  Participants  were  shown  the  current  energy  profile  of  the  country(ies)  in 
question  (expressed  as  percentage  breakdown  by  major  category — namely,  oil,  natural  gas, 
coal,  and  renewable),  as  well  as  the  expected  total  energy  requirement  for  2020,  and  were 
asked  to  propose  a  new  percentage  breakdown  for  the  2020  timeframe.  We  did  separate  mini¬ 
sessions  on  Japan,  India,  China,  the  rest  of  Asia,  and  Asia  as  a  whole. 

•  Concerning  “The  Players,”  we  conducted  a  session  called  “The  List”  (based  on  the  cable  net¬ 
work  VH-l’s  show  of  the  same  name).  Participants  were  asked  to  nominate  countries  and/or 
non-state  actors  for  the  following  “best  awards”:  Best  New  Villain,  Best  New  Ingenue,  Best 
New  Odd  Couple,  Best  New  Long-Distance  Romance,  Most  Likely  to  Get  Hitched,  and  Most 
Likely  to  Get  Dumped  (these  categories  [were]  explained  in  detail  in  later  slides). 

•  Concerning  “The  Unfolding,”  we  conducted  a  session  called  “Scenario  Flashpoints,”  where 
participants  wrote  advisory  emails  to  the  leaders  of  countries  involved  in  three  crisis  scenarios: 
oil  blockade/sanction  vignette,  gas  pipeline  disruption  vignette,  and  coal  emissions/air  pollution 
vignette. 

•  Concerning  “The  Adjustment,”  we  conducted  a  session  called  “Headlines  from  the  Future,” 
where  participants  named  four  long-term  outcome  scenarios  for  Asian  energy  developments, 
brainstormed  likely  headlines  found  along  each  pathway,  and  decided  which  Asian  states  are 
most  likely  to  end  up  in  each  scenario. 

The  second  in  the  NewRuleSets.Project  series  was  the  Foreign  Direct  Investment  decision  event. 
The  following  is  an  excerpt  from  Dr.  Barnett’s  on-line  event  report.5 

The  Foreign  Direct  Investment  event  basically  explored,  over  five  substantive  sessions,  a  rough 
“influence  net”  model  that  we  constructed  to  describe  the  key  dynamics  of  Developing  Asia’s  ability 
to  attract  outside  investment  and  that  flow’s  long-term  impact  on  the  global  economy  and  security 
environment: 
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Figure  5:  Asian  Energy  Futures  “Headlines  from  the  Future” 
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•  Concerning  “The  Blend,”  we  conducted  two  sessions.  In  the  first  session  called  “Where  Asia 
Gets  the  Money,”  participants  were  asked  to  determine  the  likely  global  pool  of  FDI  stock  for 
the  year  2010  and  then,  through  a  series  of  “drill  down”  votes,  determine  how  much  of  the  to¬ 
tal  global  pool  would  likely  end  up  in  Developing  Asia.  In  the  second  session  called  “Energy 
Case  Study,”  we  reviewed  the  findings  from  the  Asian  Energy  Futures  event  and  asked  par¬ 
ticipants  to  brainstorm  reasons  why  state  governments  in  Developing  Asia  should  play  a 
lesser  or  larger  role  in  developing  the  energy  sector. 

•  Concerning  “The  Players,”  we  conducted  one  session  called  “Build  Your  Own  Free  Trade 
Zone.”  This  voting  process  was  based  on  the  same  logic  of  “connectivity”  that  fuels  the  popu¬ 
lar  movie  trivia  game  known  as  Six  Degrees  of  Kevin  Bacon,  or  The  Kevin  Bacon  Game.  In 
this  effort,  we  asked  participants  to  build  three  separate  free  trade  zones,  and  then,  by  com¬ 
paring  the  three  groups,  drew  some  conclusions  about  which  Developing  Asian  states  offer 
the  greatest  financial  connectivity  to  the  region  as  a  whole. 

•  Concerning  “The  Unfolding,”  we  conducted  two  sessions.  In  the  first  session  called  “Pick 
Your  Dream  Investment  Partner,”  we  had  participants  brainstorm  ideas  about  what  makes 
Developing  Asia  more  or  less  attractive  as  a  target  for  FDI,  as  well  as  what  makes  the  U.S., 
the  European  Union  and  Japan  more  or  less  attractive  as  Developing  Asia’s  sources  of  FDI. 
This  session  was  based  on  the  1960s  American  television  game  show  called  The  Dating 
Game  (Figure  6).  In  the  second  session  called  “Scenario  Flashpoint,”  we  examined  how  a  fu¬ 
ture  collapse  of  the  North  Korean  regime  would  impact  the  region’s  overall  FDI  climate,  with 
our  participants  writing  advisory  emails  to  various  involved  political  leaders. 

•  Concerning  “The  Adjustment,”  we  conducted  one  session  called  “Rule  Set  Scenarios.”  In  this 
voting  process  we  asked  the  participants  to  name  and  populate — using  “Headlines  from  the 
Future” — a  quartet  of  long-term  FDI  climate  scenarios  for  Developing  Asia. 


Question  1: 

“Contestants,  tell  me  about  vour  best  feature  ... 

<  *• 


Figure  6:  The  Foreign  Direct  Investment  Dating  Game 
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Seven  “contestants”~all  threats  to  Developing  Asia’s  ecosystem 

•  We  rank-order  them  by  voting  one  off  over  six  sequential 

votes  spaced  by  focused  discussion 

•  In  each  vote,  we  ask  “What  am  I  least  worried  about 
surviving  as  Developing  Asia  grows?” 

Six  discussion  sessions,  or  “challenges”  will  unfold  over  schedule 
that  you  will  decide  in  initial  rank-ordering  vote 

•  In  this  first  vote,  we  ask,  “How  should  this  conversation 

proceed?” 


Figure  7:  Environmental  Survivor 


The  third  event  of  the  NewRuleSets.Project  was  the  Asian  Environmental  Solutions  Event.  This  event 
explored  the  impact  that  environmental  issues  might  have  on  future  Asian  economic  security.  During 
this  event  the  project  director,  Tom  Barnett,  forced  event  participants  to  make  “impossible”  choices 
by  deciding  which  environmental  crisis  they  would  choose  to  ignore  first.  To  guide  the  participants 
in  making  these  choices.  Dr.  Barnett  hosted  a  game  based  on  the  reality  television  show  Survivor. 
For  this  game,  the  “contestants”  were  the  various  environmental  crises  (e.g.,  global  climate  change, 
shortages  of  fresh  water,  loss  of  tropical  forests/extinction  of  species,  etc.)  and  the  “tribal  council” 
voted  off  each  “environmental  crisis”  based  on  which  crisis  was  least  important  to  future  Asian  eco¬ 
nomic  security  (Figure  7).  As  this  series  of  votes  progressed  it  became  increasingly  difficult  for 
event  participants  to  decide  which  of  the  equally  catastrophic  possibilities  to  ignore.  During  the  dis¬ 
cussion  “challenges”  (these  provided  discussion  guidelines)  between  each  vote,  participants  voiced 
their  concerns  and  made  the  case  to  other  “tribal  council  members”  why  their  particular  “pet  envi¬ 
ronmental  concern”  should  not  be  ignored.  The  result  of  this  exercise  was  a  well-informed  list  of 
environmental  priorities  that  this  group  of  experts  felt  the  global  community  should  deal  with  (and 
which  should  be  dealt  with  first),  as  they  would  eventually  affect  developing  Asia’s  economic  security. 

Later  in  the  Asian  Environmental  Solutions  Event,  participants  were  asked  how  they  would  spend 
a  limited  amount  of  funds  to  address  three  specific  environmental  concerns.  To  make  considering 
this  allocation  of  funds  more  interesting  Tom  Barnett  asked  the  questions  using  a  version  of  the 
game  show  Jeopardy.  He  explained  the  process  using  the  following  two  slides  (Figures  8  and  9): 
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Hqw  we’re  going  to  play 


♦  3  issues  .will  be  voted 

* 

-  Land:  deforestation 

-  Water:  shortages  (potable  and  agriculture) 

-  Air:  CQ2  emissions 

♦  For  each  vote,  you  are  given  $3000  to  distribute 
along  each  column,  where  the  answers  are  already 
placed 

-  Can  do  a  straight  rank-ordering  of  $200,  $400,  $600, 
$800  and  $1000 

-  Or  you  can  spread  money  according  to  your  preferences 

♦  Question  we  ask  is:  “If  you’re  placing  the  bet,  how 
do  you  spread  your  money  around?” 


Figure  8:  How  to  play  Environmental  Jeopardy 

All  of  these  participant  brainstorming  and  voting  sessions  were  captured  by  the  GroupSystems 
software  program  for  our  subsequent  analysis,  along  with  our  notes  of  the  accompanying  discus¬ 
sions.  Collectively,  this  material  forms  the  basis  for  the  analysis  we  present  in  the  NewRule- 
Sets.Project  reports. 
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Figure  9:  Environmental  Jeopardy 
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In  a  nut  shell 


•  The  Decision  Strategies  Department  has  found  that  very  effective  research  can  be  conducted 
for  a  variety  of  decision  makers  using  the  collaborative  approach  we  have  developed.  ‘Talk¬ 
ing  to  Experts”  in  a  manner  that  makes  the  time  they  devote  to  assisting  in  your  research  in¬ 
teresting,  not  only  makes  your  research  more  effective,  but  also  makes  the  whole  collabora¬ 
tive  research  event  enjoyable  for  all  concerned.  Events  that  are  enjoyable,  are  richer  in  their 
content  and  more  palatable  to  the  type  of  senior/recognized  experts  you  seek  to  have  partici¬ 
pate  in  your  research. 

•  Attracting  diverse  event  attendees  yields  a  wide  variety  of  ideas  and  approaches  that  will  help 
broaden  your  research  efforts. 

•  GroupSystems  collaboration  complements  discussions  and  briefings  and  suits  our  data  gath¬ 
ering  aims,  but  consumes  a  minority  of  the  time  the  participants  spend  in  a  decision  event. 

Contrary  to  popular  belief,  with  work  and  some  innovative  techniques,  it  is  possible  to  conduct  ef¬ 
fective  collaborative  research  and  have  fun  at  the  same  time. 


14 


Notes 


1.  With  Inputs  from  Dr.  Tom  Barnett,  Professor  Bradd  Hayes,  Mr.  Gregg  Hoffman,  Professor  Hank  Kamradt,  Dr.  Lawrence  Modisett, 
and  Ambassador  Paul  Taylor 

2.  GroupSystems  is  collaborative  groupware  from  GroupSystems.com 

3.  Naval  War  College  Review,  Summer  2001,  Vol.  LIV,  No.  3  (excerpt  in  this  article  contains  minor  edits  from  the  NWC  Review  gal¬ 
ley  proof).  Ambassador  Taylor  works  with  the  Naval  War  College’s  Decision  Strategies  Department,  where  he  has  directed  studies 
on  South  Asia  proliferation  and  Korean  futures. 

4.  Asian  Energy  Futures  Decision  Event  Report  I  (DSD  Report  00-6  revised)  published  April  2001. 

5.  The  On-line  Report  can  be  found  at:  http://www.nwc.navy.mil/newrulesets/FDIreport.htm 
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